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1 Introduction 

This paper is devoted to a class of two-player zero-sum stochastic differential game in which 
the players have different information on the payoff. In this basic model, the terminal cost 
is chosen (at the initial time) randomly among a finite set of costs {gij, i G {1, . . . , /}, j G 
{1, . . . , J} }. More precisely, the indexes i and j are chosen independently according to a 
probability p ® q on {1, . . . , 1} x {1, . . . , J}. Then the index i is announced to the first 
player and the index j to the second player. The players control the stochastic differential 
equation 

dXs = b{s,Xs,Us,Vs)ds + a{s,Xs,Us,Vs)dBs, s € [t,T], 
Xt = X, 

through their respective controls (ug) and (vs) in order, for the first player, to minimize 
E[gij{XT)] and, for the second player, to maximize this quantity. Note that the players 
do not really know which payoff they are actually optimizing because the first player, for 
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instance, ignores which index j has been chosen. The key assumption in our model is that 
the players observe the evolving state {Xg). So they can deduce from this observation the 
behavior of their opponent and try to derive from it some knowledge on their missing data. 

The formalization of such a game is quite involved: we refer to the second section of 
the paper where the notations are properly defined. In order to describe our results, let us 
introduce the upper and lower value functions and of the game: 

V^{t,x,p,q) = inf sup J^''^{t,x,a, 0), 

V^{t,x,p,q)= sup inf J^''^{t, x, a, (3). 

where J^''^{t, x, a, 13) is the expectation under the probability q oi the payoff associated 
with the strategies a = (aj)ie{i,...,/} ^'^^ P = if^j)j&{i,---,J} °f players. The strategy a 
takes into account the knowledge by the first player of the index i while $ takes into account 
the knowledge of j by the second player. Our main result is that, under Isaacs'condition, 
the two value functions coincide: = . Moreover, V := = V~ is the unique 
viscosity solution in the dual sense of some second order Hamilton-Jacobi equation. This 
means that 

(i) V is convex with respect to p and concave with respect to q, 

(ii) the convex conjugate of V with respect to p is a subsolution of some Hamilton-Jacobi- 
Isaacs (HJI) equation in the viscosity sense, 

(iii) the concave conjugate of V with respect to g is a supersolution of a symmetric HJI 
equation, 

(iv) Y{T,x,p,q) = Y.i,jPi<lj9ij{x) where p = (pi)ie{i,...,/} and q = (gi)je{i,...,j}. 

We strongly underline that in general the value functions are not solution of the standard 
HJI equation: indeed V does not satisfy a dynamic programming principle in a classical 
sense. 

An important current in Mathematical Finance is the modeling of insider trading (see for 
example Amendinger, Becherer, Schweizer p] or Corcuera, Imkeller, Kohatsu-Higa, Nualart 
[7] and references therein). The basic question studied in these works is to evaluate how 
the addition of knowledge for a trader — i.e., mathematically, the addition to the original 
filtration of a variable depending on the future — shows up in his investing strategies, and 
an important tool is the theory of enlargement of filtrations. Our approach is completely 
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different. Indeed, what is important in our game is not that the players have "more" 
information than what is contained in the filtration of the Brownian motion, but that their 
information differs from that of their opponent. In some sense we try to understand the 
strategic role of information in the game. 

The model described above is strongly inspired by a similar one studied by Aumann 
and Maschler in the framework of repeated games. Since their seminal papers (reproduced 
in [3] ) , this model has attracted a lot of attention in game theory (see [H] , [l3] , [E] , [E] ) . 
However it is only recently that the first author has adapted the model to deterministic 
differential games (see [5], [6]). 

The aim of this paper is to generalize the results of [S] to stochastic differential games 
and to game with integral payoffs. There are several difficulties towards this aim. First the 
notion of strategies for stochastic differential games is quite intricated (see [12] , |14j). For 
our game it is all the more difficult that the players have to introduce additional noise in 
their strategies in order to confuse their oponent. One of the achievements of this paper is 
an important simplification of the notion of strategy which allows the introduction of the 
notion of random strategies. This also simplifies several proofs of [5]. Second the existence of 
a value for "classical" stochastic differential games relies on a comparison principle for some 
second order Hamilton- Jacobi equations. Here we have to be able to compare functions 
satisfying the condition (i,ii,iv) defined above with functions satisfying (i,iii,iv). While for 
deterministic differential games (i.e., first order HJI equations) we could do this without too 
much trouble (see [5]), for stochastic differential games (i.e., second order HJI equations) 
the proof is much more involved. In particular it requires a new maximum principle for 
lower semicontinuous functions (see the appendix) which is the most technical part of the 
paper. 

The paper is organized in the following way: in section 2, we introduce the main nota- 
tions and the notion of random strategies and we define the value functions of our game. 
In section 3 we prove that the value functions (and more precisely the convex and concave 
conjugates) are sub- and supersolutions of some HJ equation. Section 4 is devoted to the 
comparison principle and to the existence of the value. In Section 5 we investigate stochas- 
tic differential games with a running cost. The appendix is devoted to a new maximum 
principle. 
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2 Definitions. 



2.1 The dynamics. 

Let T > be a fixed finite time horizon. For {t,x) € [0, T] x IR", we consider tlie following 
doubly controlled stochastic system : 

dXs = b{s,Xs,Us,Vs)ds + a{s,Xs,Us,Vs)dBs, se[t,T], 

[2.1) 

Xt = X, 

where i? is a d-dimensional standard Brownian motion on a given probability space {il., P). 
For s € [t, T], we set 

Tt.s = <y{Br - Bt,r G [t,s\}yv, 

where V is the set of all null-sets of P. 

The processes u and v are assumed to take their values in some compact metric spaces 
U and V respectively. We suppose that the functions b : [0, T] x IR" x U x V ^ IR" and 
a : [0, T] X IR" x U x V ^ IR"^'^ are continuous and satisfy the assumption (H): 

(H) b and a are bounded and Lipschitz continuous with respect to {t,x), uniformly in 
{u,v) gU xV. 

We also assume Isaacs' condition : for all {t,x) £ [0, T] x IR"', p £ IR", and all A £ Sn 
(where Sn is the set of symmetric n x n matrices) holds: 

inf„sup^{< b{t,x,u,v),p > +^Tr{Aa{t,x,u,v)a*{t,x,u,v))} = 

sup^ inf„{< b{t,x,u,v),p > +^Tr{Aa{t,x,u,v)a*{t,x,u,v))} 

We set H{t, x,p, A) = inf^ supj,{< b{t, x,u,v),p > +^Tr{Aa{t, x, u, v)a*{t, x, u, v))}. 

For t G [0,T), we denote by C([t, T],IR") the set of continuous maps from [t,T] to IR". 

2.2 Admissible controls. 

Definition 2.1 An admissible control u for player I (resp. II) on \t, T] is a process taking 
values in U (resp. V), progressively measurable with respect to the filtration {J^t.s,s > t). 
The set of admissible controls for player I (resp. II) on [t,T] is denoted bylA{t) (resp. V{t)). 

We identify two processes u and u in U{t) if P{u = u a.e. in [t, T]} = 1. 

Under assumption (H), for all (t, x) G [0, T] x IR" and (u, G U{t) x V(t), there exists 
a unique solution to ()2.ip that we denote by x*'^'**'". 
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2.3 Strategies. 

Definition 2.2 A strategy for player I starting at time t is a Borel-measurahle map a : 
[t,T] X C([t,r],]R") U for which there exists 6 > such that, Vs G [t,T],f,f' G 
C{[t,T],JR^), iff = f on [t,s], then a{- , f) = a{- , f) on [t,s + 6]. 

We define strategies for player II in a symmetric way and denote by A{t) (resp. B{t)) the 
set of strategies for player I (resp. player II). 

We have the following existence result : 

Lemma 2.1 For all {t,x) in [0,T] x IR", for all (a, (3) G A{t) x B{t), there exists a unique 
couple of controls {u,v) € ll{t) x V{t) that satisfies P—a.s. 

iu,v) = (a(-,X*'"'"'''),/3(-,X.*'-'"''^)) on [t,T]. (2.3) 

Proof: The controls u and v will be built step by step. Let 5 > be a common delay for a 
and p. We can choose 6 such that T = t + N6 for some G IN*. 

By definition, on [t,t + 5), for all / G C([t, T], IR"), a{s,f) = a{s,f{t)). Since, for all 
{u,v) G U{t) X V{t), x^*'^'"'''' = X, the control u is uniquely defined on [t,t + 5) by 

Vs G [t,t + 5),u{,s) = a{s, x). 

The same holds for v, what permits us to define the process x.*'^'"'^ on [t, t + (5) as a solution 
of the system (j2.ip restricted on the interval [t, t + 5). 

Now suppose that n, v and x.*'^'"'^ are P—a.s. defined uniquely on some interval [t, t + k6), 
k G {1, . . . , N — 1}. This allows us to set, 

Vs e[t + k6,t + {k + 1)6), Us = a(s,X.*'"'"'''''),i;, = /3(s, X*'"'"''^'), 

where 

/ fc fcx / (^'^) [t,t + k6) 
[u ,v ) = < 

[ {uo,vo) else, 
for some arbitrary (uo,fo) G U{t) x V(t). 

Considering x.*'^'" as a random variable with values in the set of paths C{[t, T), IR"), it is 
clear that the map (s,u;) — > Us{lo) (defined on [t + k6, t + {k + 1)5) x 0) as the composition of 
the Borel measurable application a with the map {s,uj) — > (s,X.*'^'" ''^ (i-^))) is a process on 
+ k5, t + {k + 1)6) with measurable paths. Further, the non anticipativity of a guaranties 
that, for all s G [t + k6,t + {k + 1)6), Us is JF^^j+fe^-measurable and the process ^i|[t,t+(fc+i)<5) 
is (J^t,s )-pi'ogressively measurable. The same holds of course for w![f,t+(fc+i)5)- 
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With {u,v) defined on [t,t + (A: + 1)6), we can now define the process x. '^'"'^ up to time 
t + (fc + 1)6. This completes the proof by induction. □ 

We denote by X*'^'"'^ the process x*'^'"'^, with {u,v) associated to (a,/?) by relation 

(1231). 

In the frame of incomplete information it is necessary to introduce random strategies. 
In contrast with [5] and [6], where the random probabilities are supposed to be absolutely 
continuous with respect to the Lebesgue measure, play a random strategy will consist here 
to choose some strategy in a finite set of possibilities, i.e. the involved probabilities are 
finite. It is not clear if this assumption is more realistic nor if the notation will be lighter, 
nevertheless this alternative allows us to avoid some technical steps of measure theory, in a 
paper that is already technical enough. 

Notation: For R E IN*, let A{R) be the set of all (ri,...,rR) E [0,1]-^ that satisfy 

We define a random strategy a for player I by a = {a^, . . . a^; r^, . . . , r^), with R E IN*, 
{a\...a^) E {A{t))R, {r\...,r^) E A{R). 

The heuristic interpretation of a is that player I's strategy amounts to choose the pure 
strategy a'^ with probability r^. 

We define in a similar way the random strategies for player II, and denote by Ar{t) (resp. 
Br{t)) the set of all random strategies for player I (resp. player II). 

Finally, identifying a E A{t) with (q; 1) E Ar{t), we can write A{t) C Ar{t), and the same 
holds for B{t) and Br{t). 



2.4 The payoff. 

Fix /, J E IN*. 

For 1 < i < /, 1 < J < J, let gij : IR" — > IR be the terminal payoffs. We assume that 

For 1 < i < /, 1 < J < J, gij are Lipschitz continuous and bounded. (2-4) 

For (p, q) E A(/) x A( J), with p = (pi, . . . ,pi), q = (gi, . . . qj), we denote with a hat the 
elements of {Ar{t))^ (resp. {Br{t))'^): a = (ai, . . . ,a/), /3 = (/3i, . . . 
We adopt following notations : 
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For fixed G {1, . . . ,/} x {1, . . . , J} and strategies (a,/3) G A{t) x B{t), the payoff of 
the game with only one possible terminal payoff function gij will be denoted by 

Jij{t,x,a,(3)=E[gij{X'/^''^% 

Now let (a, (3) G Ar{t) x be two random strategies, with a = (a^, . . . , a^; r^, . . . , r^) 

and (3 = . . . , s^, . . . , s"^). The payoff associated with the pair (a, /3) G ^r(i) x Br{t)), 
is the average of the payoffs with respect to the probability distributions associated to the 
strategies: 

R S 
k=l 1=1 

Further, for p G A(/), j G {1, . . . , J}, a G {Ar{t))^ and /3 G Br{t) we will use the notation 

I I 

i=l i=l k,l 

A symmetric notation holds for a G Ar{t) and (3 G Finally, the payoff of the 

game is, for {a, (3) G (A(i))^ x {Br{t)y , p G A(/), g G A(J), 

/ J 

JP'''(t,a;,a,/3) = '^'^piqjJij{t,x,ai,f3j). 
i=i j=i 

The reference to (t, x) in the notations is dropped when there is no possible confusion : we 
will write Jij{a, Jij{a, /?),.... 

We define the value functions for the game by 

V^{t,x,p,q) = inf^g(_4^(i))/ sup^g^^ ^^^^j JP''i{t,x,a, (3), 
V-{t,x,p,q) = sup^g(g^(^))j inf^g(_4^(i))7 JP'i{t,x,a, (3). 

Again we will write V~^{p, q) and V~{p, q) if there is no possible confusion on [t, x). 

The following lemma follows easily from classical estimations for stochastic differential 
equations : 

Lemma 2.2 and are hounded, Lipschitz continuous with respect to x,p,q and 
Holder continuous with respect to t. 

Following [3] we now state one of the basic properties of the value functions. The 
technique of proof of this statement is known as the splitting method in repeated game 
theory (see [3], [IB]). 
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Proposition 2.1 For all {t,x) G [0,T] x IR", the maps {p,q) V'^{t,x,p,q) and {p,q) 
V~{t,x,p,q) are convex in p and concave in q. 

Proof: We only prove the result for the proof for V~ is the same. First can be 
rewritten as 



It follows that is concave in q. 

Now fix g G A(J) and let p,p' G A(7) and a G (0, 1). Without loss of generality we can 
assume that, for alH G {1, . . . , /}, Pi and are not simultaneously equal to zero. 
We get a new element of A (J) if we set = ap + (1 — a)p'. For e > 0, let a G {Ar{t)y be 
e-optimal for V'^{p,q) (resp. a' G {Ar{t)y e-optimal for V'^{p',q)). 
We define a new strategy a" = (a", . . . , a") by 



J 




(^i ' • • • ? 5 ? • • • ? 



f ;(rf)\...,(rf)(«+«')),zG{l,...,/} 




for ke{l,...,R}, 



p7 i 



for ke{R+l,...,R + R'} 



(it is easy to check that a" G {Ar{t))^). 



This means that, for aU /3 G {Br{t)y , 




i=l fe=l k=l 



Thus 



sup < a sup JP''i{aJ) + {l-a) sup J^'''«(a',/3). 

/3e(Br-{t))-' $e{Br{t)y $e{Br{t)y 



It follows by the choice of a and a' that 



< aV+{p, q) + {l- a)V+{p', q). 



□ 
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3 Subdynamic programming and Hamilton-Jacobi-Bellman 
equations for the Fenchel conjugates. 



Since and V~ are convex with respect to p and concave with respect to q, it is natural to 
introduce the Fenchel conjugates of these functions. For this we use the following notations. 
For any w : [0, T] x IR" x A(/) x A( J) — > IR, we define the Fenchel conjugate w* of w with 
respect to p by 



w*{t,x,p,q) = sup {{p,p) - w{t,x,p,q)}, (t, g) G [0, T] x IR" x IR^ x A( J). 



w*{t,x,p,q) = sup{{p,p) -w{t,x,p,q)}, {t,x,p,q) G [0,r] X x A(/) X A(J). 



It is well known that, if w is convex, we have (w*)* = w. 

We also have to introduce the concave conjugate with respect to g of a map w : [0, T] x 



IR" X A(/) X A(J) ^ IR: 

w^{t,x,p,q)= inf {{q,q) - w(t,x,p,q)}, {t,x,p,q) £ [0,T] xWC x A{I) xJR-^ . 



We use the following notations for the sub- and superdifferentials with respect to p and q 
respectively: if w : [0,T] x IR" x IR^ x A(J) IR, we set 

dpw{t,x,p,q) = {p e TR^ , w{t,x,p,q) + {p,p - p) < w{t,x,p',q) Mp S IR'^} 

and if w : [0, T] x IR" x A(/) x IR-^ ^ IR 

dfw{t,x,p,q) = {q e IR"', w{t,x,p,q) + {q,q' - q) > w{t,x,p,q') Vg' G IR"^}. 

In this chapter, we will show that V'^'^ and V~* satisfy a subdynamic programming 
property. This part follows several ideas of [TO], [TT] . 

Lemma 3.1 (Reformulation ofV^*) 

For all {t,x,p,q) G [0, T] x IR" x x A(J), we have 



pGA(/) 



For w defined on the dual space [0,T] x IR" x IR'' x A(J), we also set 



gGA(J) 



V-*{t,x,p,q) 



inf sup max 

$eiBr{t)y' a<^A(t) »G{1, ...,/} 




(3.5) 



Proof. We begin to establish a first expression for V 



V-*ip,q) 



inf sup max 




(3.6) 
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(the difference witfi (|3.5p is that player I here can use random strategies.) 

Let's denote by e = e{p,q) the right hand term of (j3.6|) . First we prove that e is convex 

with respect to p : 

Fix q G A(J), p,p' G and a G (0, 1). 

For e > 0, let (3 (resp. /3')g (Sr(t))'^ be some e-optimal strategy for e{p,q) (resp. e{p',q)). 

Set = aj5 + (1 — a)j5'. 

We define a new strategy /3" G (^rCi))-^ by 

= . . . , /3f , /3f , . . . , (3f;{s^)\ is^f+''), {!,..., J}, 

with 

for A; G {!,..., S"}, 



' {l-a)sf-'' ke{S+l,...,S + S'}. 



Let a G Ar{t). Since the application (2:1,..., x/) — > max{xi,i = 1,...,/} is convex, we 
have 



max,- 



{pf - J!{a,h} = max, {a(K - (a,/3)) + (1 - a)(p: - jfiaj'))} 



< a supjjg^^.(t) maxi(pi - Jf (a, 



+ (1 - a) supjjg_4 maxi(]5,i - (a, 



< ae(p,g) + (1 - a)e(p',g) +e. 

Since e is arbitrary, we can deduce that e is convex with respect to p. 

The next step is to prove that e* = . By the convexity of e, this will imply that V~* = e. 

We can reorganize e* (p, g) as follows : 

e*{p, q) = sup^gjRi {eLi KPi + suP/3e(B.(t))-^ infaeyi,{t) ^^^i'e{i,...,i}{J!^ia, (3) - pe}^ 

= sup^g(B,.(t)).7 supj^gj^/ E[=iPimini'e{i,...,/} {infae^.(i) '/^^(a,/?) + {Pi - Pi')] 

The supremum over p G IR^ is attained for pi/ = inf^g^^(^) >/jv(a, P) and we get the claimed 
result. 



Finally, to get (13.51) . it remains to show that player I can use non random strategies. 
Indeed, writing V~* as in ()3.6p and since A{t) C Ar(t), it is obvious that the left hand side 



10 



of (j3.5p is not smaller than the right hand side. 
Concerning the reverse inequality, we can write 

suPcEG A- (t) max, jpi - Jf (a , /3) | 

< sup^giN* s^Pia\...,aR)e{Ait))R,ir^,...,rR)eAiR) Efc=i ^'^ max^ jpi - (a^^, /?)} 

< sup^gjN* sup(^i^...^^_R)gA(i?) Efc sup„g^(t) maxi |pi - J/(a,/3)}. 
The result follows after one recalls that '}2k=i ^'^ = 1- 



□ 



Proposition 3.1 (Subdynamic programming for V *) 

For all <to <ti <T,xo e IR",p eM^,q £ A(J), it holds that 

V-*{to,xo,p,q)< inf sup g)]. 

/3GB(io)ae^(f(,) 

Proof: Set V^* {to,ti, xo,p, q) = mf^^sito) s^PaeA{to) ,p,q)]. 
For e > 0, let (3'^ G -6(^0) be e-optimal for V{'* {tQ,ti, xo,p, q), and, for all x € IR", let 
G (^rl^i))-^ be e-optimal for V-*{ti ,x,p,q). By the uniformly Lipschitz assumptions for 
the parameters of the dynamics, there exists R> such that, for all a G A{tQ), 

p^^to,xo,a,(3^ ^B{xo,R)] > 1-e, 
where B(xq, R) denotes the ball in IR" of center xq and radius R. 

Remark that and V~* are uniformly Lipschitz continuous in x. This implies that we can 

find r > such that, for any x S IR" and y € B{x,r), j3^' is 2e-optimal for V^* {ti,y,p, q). 

Now let xi, . . . , XA/ G IR"- such that U^^iB{xm, §) 3 B{xo, R). 

Set (3"^ = Z?^™ for m = 1, . . . , M and choose some arbitrary G (Briti))"^ . 

Each Z?™ is detailed in the following way: 

with 
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Let 5 be a common delay for f5^ , . . . , j3^^ that we can choose as small as we need : 

2 

< (5 < ^ A (ti — to)i where C > is defined through the parameters of the dynamics by 

Va G >l(t),/3 G G [to,T], _ <C\t-t'\. 

We then have in particular, for all a G A{t) and j3 G B{t), 

pj[^to,xo,a,/3 _ ^to^xo,a,/3| > L] < e. (3.7) 

Let {Em)m=i,...,M be a Borel measurable partition of B{xo,R), such that, for all m G 
{1, . . . , M}, ^„ C B{xm, §). Set ^0 = B{xo, Rf. 

We are now able to define a new strategy for player II, (3'^ G {Br{tQ)y: 

M m,lm 



Fix j G {1, . . . , J}. For / = (/o, . . . , /a/) G 1^ := n*^^o{l> • • • > -^f }, set 4 = H^^q 
Remark that {s^-,/ G L} G A(Card(L)). 
Then, for / G L, / = (/q, • • • , /m), we define G ^(to) by 

V/GC([to,T],IR-),VtG [to,T], 

f ift G [to,ii), 



PT''"^itJ\[t^^T]) if i G and f{h - 6) e E„ 



We set /3 ■ := s'.-, / G L) G S^(to), and finally = (/?!, ... , 



For some fixed a G ^(to) and / G C([to, ^i], IR"), we define a new strategy G ^(ti) by: 
for ah t G [0,r] and /' G C([ti, T], IR"), 



a/(t,/') = a(t,/), with/(t) 



/(t) for t G [to,ti], 

f{t)-r{h) + f{h), fortG (ti,r]. 



Set X' = X*°'''°''''^' and, for m G {0,...,M}, A„ = G ^m}. Set further 

A = {\X^^ - X^^_g\ < §}. By ([3?7l) . it holds that P[A''] < e. Remark also that, on each 
A n Am, Xl_^ belongs to B{xm,r) and consequently, still on A n Am, P"^ is 2e-optimal for 
V~*{ti,XI^,p,q). 

For alH G {1, ...,/}, j G {1, . . . , J} and Z G L, we have 



m=0 



)|^,J = .A^E[g,,{X 
It follows that 

Jf{to,xo, a, P') = Y.'Li Qj T.ieL s\E[g,j{Xp^°''''^^'^ 
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And 



< E[{V-*{h,X^^,p,q) + 2e)iAn{xi^eB(.o,R)}] 

+ maXie{i,...,7}{|K| i?(xo, i?)]), 

by the choice of (/3"*, m G {1, . . . , M}) and where if is an upper bound of \g\. 

By the choice of R and with the notation K{p) = 4maxjg{i + K} + e, we get 



max. 



{i,...,i}{pi-J!{to,xo,aJ')] < E[V-*iti,XI^,p,q) + 2e]+ K{p)e 

< sup,,^(,„) q)] + 26(1 + K{p)) 

< V^-*{to,tuXo,p,q) + e{3 + 2K{p)) 

(for the last inequality, recall that was chosen e-optimal for V^*{to,ti,xo,p,q)). 

We can deduce the result. □ 



A classical consequence of the subdynamic programming principle for is that this 
function is a subsolution of some associated Hamilton-Jacobi equation. We give a proof of 
that result for sake of completeness. 

Corollary 3.1 For any {p,q) G IR^ x A(J), V~*{-, --iP^q) is a subsolution in the viscosity 
sense of 

wt + H-*{t, X, Dw, D'^w) = 0, (t, x) € (0, T) x IR", 

with 

H-* {t,x,p,A) = ^H-{t,x, -p, -A) = 

infy^vsupy^^lj{{b{t,x,u,v),p) + ^Tr{Aa{t,x,u,v)a* {t,x,u,v))}. 

Proof : For {to,xo) G [0,r] x TR"-,p eJR^,qe A(J) fixed, let ^ G C^'^ such that ^{to,xo) = 
V-*ito,xo,p,q) and, for all (s,y) G [0,T] x H", 4>{s,y) > V-*is,y,p,q). 
We have to prove that 

(ptito,xo) + H~*{to,xo,D4>{to,xo),D'^{to,xo)) > 0. 
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Suppose that this is false and consider 6 > such that 



Mto, xo) + H-*{to, xo, DcPito, XQ),D\to, xq)) <-0<O. (3.9) 

Set A{t,x,u,v) = (pt{t,x) + {b{t,x,u,v), Dcl){t,x)) + Tr{D'^(p{t,x)a{t,x,u,v)a*{t,x,u,v)). 
Since, for fixed j5, V~* is bounded, we can choose cj) such that (pt and D'^cj) are also bounded. 
It follows that, for some K > 0, we have \A{t, x,u,v)\ < K. 
Now the relation ()3.9p is equivalent to 

inf sup A(to, 2^0) ^) "w) < — ^ • 

This implies the existence of a control such that, for all u U, 

A{to,xo,u,vo) < — J. 

Moreover, since A is continuous in {t,x), uniformly in u,v, we can find R> such that, 
y{t,x) E [to,T] X IR", \t - to\ V ||x - xoll < i?,Vn G U, A{t, x,u,vo) < --. (3.10) 

Now define a strategy for player II by f3o{tJ) = vq for ah {t, f) G [to,T] x C([to, T], IR"). 
Fix e > and t £ (tQ,R). Because of the subdynamical programming (Proposition 13. 1| ). 
there exists a^^t £ -^(^o) such that 

E[V-*{h,xl'^''""''-''''°,p,q)]-V'*{to,xo,p,q) > -e{t-to). (3.11) 

Let {us,Vs) E K(to) X V(to) the controls associated to {ae^t,Po) by the relation (j2.3p and 
set X = = x*"'^"'"'". (Remark that, by the choice of /3o, (vs) is constant and 

equal to vq.) 

Now we write Ito's formula for (j){t,Xt): 



(l){t,Xt) - (t){to,xo) = fl:^^A{s,Xs,Us,Vs)ds 

+ Jl {D<Pis, Xs), b{s, Xs,Us,Vs))dBs. 

By ()3.1ip . (j3.12p and the definition of (j), we have 



(3.12) 



E[ [ A{s, Xs,Us,Vs)ds] > -e{t - to). (3.13) 

Jtn 



'to 

In the other hand, there exists a constant C > depending only on the parameters of X, 
such that 

P[\\X-xo\\,>R]<^^^^, 
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with the notation = sup^gj^g^ij ||/(s)||. 

Following ([XTUj) . this imphes that, for ah t G [to, T A (to + -R)], 



E 



t 

Ii\\X.~XQ\\t<R} I Hs,Xs,Us,Vs)ds 
to 



6. 

<-^{t-to). 



(3.14) 



By (|3.13|) and (j3.14p . we now have 



e{t-to) < E[Jl A{s,Xs,Us,v,)dsI^\\x-xo\U>R}]+E[j!A{s,Xs 

,Us,Vs)dsI{iix.-xo\\t<R}] 



< 



KC 
'W 



or, equivalently, 



e KC\ 



Since t — to ^ can be chosen arbitrarily small, we get a contradiction. 



□ 



For we have: 

Proposition 3.2 (Superdynamic programming and HJI equation for V^^) 
For all < to < ti < r, xo G IR",p G A(/), q G IR"^, it holds that 

V+\to,xo,p,q)> inf sup i^[y+tt(ti, g)]. 

l3eB{to) a&A{to) 

As a consequence, for any {p,q) G A(/) x IR'', V~^'^{-, -jPjCj) is a supersolution in viscosity 
sense of 

wt + H+*{t, X, Dw, D^w)) = 0, (t, x) G (0, T) x IR", 

where 

H+*{t, x,p, A) = -H+{t, X, -p, -A) = 

(3.15) 

sup„g[/ inf^,ey{(6(t, x, u, v),p) + ^ Tr{Aa{t, x, u, v)a*{t, x, u, v))}. 

Proof : We note that is equal to the opposite of the lower value of the game 
in which we replace gij by —gij, Player I is the maximizer and in which the respective 
roles of p and q are exchanged. Using Proposition 13.11 in this framework gives the superdy- 
namic programming principle. Now Corollary 13.11 shows that, for any {p,q) G A(I) x M'^, 
{—V^)*{-, q) = —V~^^{-, ■,p, —q) is a subsolution of 

Wt + F+(t, X, Dw, D'^w)) = 0, (t, x) G (0, T) x IR". 
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Hence V~^'^{-, ■,p, —q) is a supersolution of 

wt + H+*{t, X, Dw, D^w)) = 0, {t, x) G (0, T) x IR". 
Since this holds true for any {p,q), this proves our claim. 



□ 



4 Comparison principle and existence of a value 

In this section we first state a new comparison principle and apply it to get the existence 
and the characterization of the value. Then we give a proof for the comparison principle. 

4.1 Statement of the comparison principle and existence of a value 

Let H : [0, T] x IR'" x x cS„ x A(/) x A( J) ^ IR be continuous and satisfy 

H{s,y,^2,X2,p,q) - H{t,x,^i,Xi,p,q) > 

-UJ (16 - 61 + a\{t,x) - {s,y)\^ + b + \{t,x) - {s,y)\{l + |6| + |6I)) 

where uj is continuous and non decreasing with a;(0) = 0, for any a,b > 0, [p, q) € A(/) x 
A(J), s,t e [0,r], x,y,ii,X2 G IR" and Xi,X2 G 5„ such that 

Definition 4.1 We say that a map w : (0, T) x IR" x A(/) x A( J) ^IR is a supersolution 
in the dual sense of equation 

wt + H{t,x,Dw,D'^w,p,q) =0 (4.17) 

if w = w{t,x,p,q) is lower semicontinuous, concave with respect to q and if, for any 
C^((0,r) X ]R") function 4> such that {t,x) w*{t,x,p,q) — (j){t,x) has a maximum at 
some point (t,x) for some {p,q) G IR^ x A(J), we have 

Mt,x)-H(t,x,-D^{t,x),-D^(P{i,x),p,q)>0 ^p e dpW*{t,x,p,q) . 

We say that w is a subsolution of (|4.17|) in the dual sense if w is upper semicontinuous, 
convex with respect to p and if, for any C^((0,T) x IR") function (p such that {t,x) — 
w'^{t,x,p,q) — (f){t,x) has a minimum at some point {t,x) for some {p,q) G A(/) x IR"^, we 
have 

Mix)- H{t,x,-D(P{t,x),-D^cl){t,x),p,q) < Vg G dfw^{t,x,p,q) . 
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A solution of in the dual sense is a map which is sub- and supersolution in the dual 

sense. 

Remarks : 

1. We have proved in Corollarv 13.11 that V~ is a dual supersolution of the HJ equation 

wt + H'{t, X, Dw, D'^w) = , 

where H~ is defined by (|3.8|) . while Proposition l3.2l shows that is a dual subsolution 
of the HJ equation 

Wt + H^{t, X, Dw, D'^w) = , 
where H~ is defined by (j3.15p . 

2. The necessity to deal with a Hamiltonian H with a (p, q) dependence will become 
clear in the next section where we study differential games with running costs. 

3. An equivalent definition of the notion of dual super- or subsolution in given in Lemma 
15.31 below. 

The main result of this section is the following: 

Theorem 4.1 (Comparison principle) Let us assume that H satisfies the structure con- 
dition Let wi he a hounded, Holder continuous subsolution of ^■I'T^ in the dual sense 
which is uniformly Lipschitz continuous w.r. to q and W2 be a bounded. Holder continuous 
supersolution of ( |^. j7| j in the dual sense which is uniformly Lipschitz continuous w.r. to p. 
Assume that 

wi{T,x,p,q) < W2{T,x,p,q) y{x,p,q) G IR" x A(I) x A(J) . (4.18) 

Then 

Wi{t,x,p,q) < W2{t,x,p,q) \/{t,x,p,q) G [0,r] X M" x A(I) X A(J) . 

Remark : For simplicity we are assuming here that wi and W2 are Holder continuous 
and bounded. These assumptions could be relaxed by standard (but painfull) techniques. 
We do not know if the uniform Lipschitz continuity assumption on wi with respect to q and 
on W2 with respect to p can be relaxed. 

As a consequence we have 
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Theorem 4.2 (Existence of a value) Under assumptions (H), and h2.S\) . the game 
has a value: 

V+{t,x,p,q) = V-{t,x,p,q) \/{t,x,p,q) G (0,r) x x A(J) x A(J) . 

Furthermore = is the unique solution in the dual sense of HJI equation Ii4.1'7\ ) with 
terminal condition 

I J 

V+{T,x,p,q) = V-{T,x,p,q) = ^ V(x,p,g) € IR" x A(/) x A(J) . 

i=i j=i 

Proof of Theorem 14.21 : The Hamiltonian H defined by (12. 2p is known to satisfy 
(j4.16p (see [9j for instance). ^From the definition of and V~ we have < V^. We 
have proved in Lemma 12.21 and Proposition 12.11 that and V~ are Holder continuous, 
Lipschitz continuous with respect to p and q, convex w.r. to p and concave w.r. to q. From 
CoroUarv 13.11 we know that V~ is a supersolution of (j4.17p in the dual sense while Propo- 
sition 13.21 states that is a supersolution of that same equation in the dual sense. The 
comparison principle then states that V'^ < V~ , whence the existence and the characteriza- 
tion of the value: V'^ = V~ is the unique solution in the dual sense of HJI equation ()4.17p . □ 

4.2 Proof of the comparison principle 

The proof of Theorem 14.11 relies on two arguments: first on a reformulation of the notions of 
sub- and supersolutions by using sub- and superjets; second on a new maximum principle 
described in the appendix. 

Let us recall the notions of sub- and superjets of a function w : (0, T) x IR" — > IR: the 
subjet D'^'~w{t,x) is the set of {S,t,^x,X) G IR"+^ x 5„ such that 

w{t,x) > w{t,x) + ^t{t -t) + ix-{x -x) + ^X{x - x).{x -x) + o{\t -t\ + \x - x|^)} 

and the superjet D'^''^w is given by 

D^^+w{t,x) = -D'^-{-w){t,x) 

When w depends on other variables {{p, q) or (p, q) for instance), D^'^w and D'^'^w always 
denote the sub- and superjets with respect to the {t, x) variables only. For w = w{t, x,p, q), 
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we set 



D"^' w{t,x,p, 



{(.t,(.x,X) G ]R"+^ X Sn , 3{tn,Xn,Pn,qn) ^ {t,X,p,q), 

With 



We use a symmetric notation for D'^'+w{t,x,p,q). 

The following equivalent formulation of the notion of sub- and supersolution is standard 
in viscosity solution theory, so we omit the proof: 

Proposition 4.3 A map w : (0, T) x IR" x A(I) x A( J) ^ IR is a supersolution of equation 
Ii4.1'7\ ) in the dual sense if and only if w = w{t,x,p,q) is lower semicontinuous, concave 
with respect to q and if, for any {i,x,p,q) and any {(,t,^,x,X) G D'^'^w*{i,x,p,q) we have 

- H{i,x, -ix, -X,p,q) > Vp G dpW*{i,x,p,q) . 

Symmetrically w is a subsolution of ( [^.iTp in the dual sense if and only if w is upper 
semicontinuous, convex with respect to p and if, for any (i,x,p,q) and any {S^t,S,x,X) G 
D'^'^w^{i,x,p,q) we have 

6 - H{i, X, -e^, -X,p, g) < Vg G d^w\t, x,p, q) . 

Proof of Theorem 14.11 : Let us assume that 

sup {wi — W2) > . 

t,x,p,q 

Since wi and W2 are Holder continuous and bounded, classical arguments show that 



f , , , , . [(i, x) — (s, y)P Q 2 I i2nn 

Me,^,Q := sup iwi{t,x,p,q)-W2{s,y,p,q)-{ ^ + \y\ )) + Vt 

t,x,s,y,p,q ze Z 

is finite and achieved at a point (t, x, s, y,po, qo)- One can also show that 

lim Mf^^ri,a = sup {wi — W2) (4.19) 

€,ri,a^O+ ' ' t,x,p,q 

and that 

\{t,x) — {s,y)\ 2 |-|2 ^ oj,^ /'/I on\ 

5 , a\x\ , a\y\ < 2Moo (4.20) 

where Moo = l^u^iloo + |u^2|oo- Using (j4.18p and the Holder continuity of wi and W2 shows 
that t < T and s < T as soon as e, rj and a are small enough. 
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^From the maximum principle (Theorem 16.11 stated in the Appendix), there are {p,q), 
{p, q) and Xi,X2 € 5„ such that 

p G dTwl{s,y,p,q), q e d^wl{i,x,p,q) , 

jt-s) jx-y) - Y \ ^ 1W= h '^ 

( ^ hr/, ^ aj:,Ai)ED^' w\[s,x,p,q) , 



(^^, - ay, ^2) e D^^+wl{-s,y,p, 



and 



Since is a subsolution of ()4.17|) in the dual sense and q G d'^W2{t,x,p,q), Proposition 
14.31 states that 

- ^-^ - H (^,x,^^^^ + ax,-Xi,p,^ < . (4.22) 

In the same way, since W2 is a supersolution of (j4.17p in the dual sense andp G dpW2{s, y,p, q). 
we have 

^""^"^ i?f.-,y-,-^^+ay,-X2,p-,g-') >0, (4.23) 



eye 

Using the structure condition (|4.16p on H, and plugging estimates (|4.19p . (j4.2Up and 
()4.2ip into (j4.22p and (|4.23p yields to a contradiction for e, a and rj sufficiently small as in 



5 Games with running cost 

We now investigate differential games with asymmetric information on the running cost and 
on the terminal cost. The framework is basically the same as before. At the initial time, 
the cost (now consisting in a running cost and a terminal one) is chosen at random among 
I X J possible costs. The index i is announced to Player I while the index j is announced 
to Player II. Then the players play the game in order, for Player I to minimize the payoff 
and for Player II to maximize it. 

In this section we keep the same terminology and the same notations as in the previous 
part. There is however a main difference: as we shall see later, in a game with a running 
cost, each player needs the knowledge of this running cost to build his strategy. Since 
we assume that the running cost depends on the control of both players, this means that 
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the players have to observe the control of their opponent. This was not the case of the 
game before where the players only observed the state of the system. For this reason we 
have to change the notion of strategy: in this section the notion of strategies introduced in 
Definition 12.21 is replaced by the following one: 

Definition 5.1 A strategy for player I starting at time t is a Borel-measurahle map a : 
[t,T] X C([t,T],]R") X L'^{[t,T],V) U for which there exists 6 > such that, for all 
s G [t,T], f,f G C([t,T], IR") andg,g' G L^{[t,T],V), if f = f and g = g' a.e. on [t,s], 
then a{-,f,g) = a{-,f,g') on [t,s + 6]. 

We define strategies for player II in a symmetric way and denote by A{t) (resp. B{t)) the 
set of strategies for player I (resp. player II). 

We define random strategies as before (but with the modified notion of strategies) and still 
denote by Ar{t) (resp. Br{t)) the set of random strategies for player I (resp. player II). 
We have an analogue of Lemma 12.11 : 

Lemma 5.1 For all {t,x) in [0,T] x IR", for all {a, (3) G A{t) x 13{t), there exists a unique 
couple of controls {u,v) G ll{t) x V{t) that satisfies P—a.s. 

{u,v) = (a(•,X*'^'"'^^;.),/3(•,X*'^'"'^n.)) a.e. on [t,T]. (5.24) 

One can easily check that the results of the previous parts (i.e., 12.21 Proposition 12. 1^ 
Corollary 13.11 and Proposition 13. 2p still hold true with the modified notion of strategy. In 
particular, the game with terminal payoff studied before has a value. 

Let us fix /, J G IN. For 1 < i < I and 1 < j < J we consider the terminal cost gij : IR" — > IR 
and the running cost lij : [0, T] x IR" x [/ x 1/ — > IR on which we do the following assumptions: 

For any 1 < i < I and 1 < J < J, iij and gij are continuous in all variables, 
uniformly Lipschitz continuous with respect to x and bounded. 

For fixed G {1, . . . , /} x {1, . . . , J} and strategies (a, f3) G A{t) x B{t), we set 
J^jit,x,a,(3) = E[j%,,{s,X'f'"^^as,[3s)ds + g,,{X'/'^^^)] , 

where as before (a, (3) denotes the unique pair of controls such that (j5.24p holds. 
The payoff of two random strategies (a, /3) G Ar{t) x Br{t), with a = (a^, . . . , a^; r^, . . . , r^) 
and^= (/3\...,/3'^;s\...,s^), is the average of the payoffs with respect to the probability 
distributions associated to the strategies: 

R S „T 

J,,(t,x,a,:5) = ^^rVi?[/ ^,,(s,X*'-'"'-/^\aJ,/3i)d. + 5ii(X^^'"''^')]. 

k=l 1=1 



21 



Finally, the payoff of the game is, for (a,/3) = {{ai)i<i<i, {Pj)i<j<j) ^ {•^r{t))^ x {^r{i))'^ , 

I J 

JP''^{t,x,a,i3) = '^'^piqjJijit,x,ai,f3j). 
i=i j=i 

We define the value functions for the game with running cost as before by 

V+{t, x,p, q) = inf^g(_4^(t))i sup^gg^(^))j JP'^it, X, a, 
V'{t,x,p,q) = sup^g(g^(^))j inf^g(_4^(i))7 JP'i{t,x,a, (3). 

In our game with running cost, Isaacs' assumption takes the following form: for all 
(t, x) G [0, T] X IR", {p, q) G A(/) x A( J), ^ G IR", and all A G 

inf„supj,{< b{t,x,u,v),(, > +^Tr{Aa{t, x,u,v)a* {t, x,u,v)) - Y^-j iij{t, x,u,v)piqj} = 

sup^ inf„{< b{t,x,u,v),^ > +lTr{Aa{t,x,u,v)a*{t,x,u,v)) - iijit,x,u,v)piqj} 

(5.26) 

We set 

H{t, X, ^, A,p, q) = inf„ sup^{< b{t, x,u,v),^> 

+ lTr{Aa{t,x,u,v)a*{t,x,u,v)) - Y^ij iijit, x,u,v)piqj} . 

Theorem 5.2 Assume that (H), 115. 25\) and i5.26]) hold. Then the game has a value: 
= , which is the unique solution in the dual sense of 

wt + H{t,x,Dw,D^w,p,q) = (5.27) 

with terminal condition 

I J 

V+{T,x,p,q) = V~{T,x,p,q) = Y,^Pi(lj9ij{x) ^{x,p,q) G x A(/) x A(J) . 

i=i j=i 

In order to prove Theorem 15.21 it will be convenient to have the following equivalent 
definition of dual solutions of the Hamilton- Jacobi equation (j5.27p : 

Lemma 5.3 Let w : [0,T] x IR^ x A(/) x A(J) i-^ IR 6e lower-semicontinuous, uniformly 
Lipschitz continuous with respect to p and concave with respect to q. Then the following 
statements are equivalent: 

(i) w is a dual supersolution of {5.2'T\j . 

(a) for any (p, q) G IR^ x A(J), for any test function (p = (f){t, x) such that 

{t, X, p) ^ w{t, X, p, q) - (l){t, x) - {p, p) 
has a global minimum at some point {t,x,p) G [0,T) x IR^ x A(I), we have 

(l)tit,x) + H{t,x,Dc^{t,x,p),D^cj){t,x,p),p,q) <0. (5.28) 
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Remark : A symmetric statement holds for dual subsolutions. 



Proof of Lemma 15.31 : Let us assume that w is a supersolution and let G C^, 
{p,q) E X A(J) such that 

{t,x,p) ^ w{t,x,p,q) - (l){t,x) - {p,p) (5.29) 

has a global minimum at some point {t, x,p) G [0, T) x IR^ x A(/). We note that this implies 
that p G dpW*{t,x,p,q). Moreover, taking the supremum over p in (j5.29p . we have that 
{t,x) — > —(l){t,x) — w*{t,x,p,q) has a global minimum at {t,x). Since w* is a subsolution 
of the dual equation, we get 

-(l)t + H*{t,x,-D(l), -D'^(l),p,q) > 

at (t, x), because p G dpW*{t,x,p,q). Whence inequality ()5.28p . 

Conversely let us assume that w satisfies (ii). Let (/) he a C'^ test function such that 
{t,x) w*{t,x,p,q) — 4>{t,x) has a maximum at some point {t,x) G (0, T) x IR^ for 
some {p,q) G IR^ x A (J). Without loss of generality we can assume that this maximum 
is a global one. Let p G d^w*{t,x,p,q). ^From the definition of w* , we also have that 
{t,x,p) {PiP) — w{t,x,p,q) — (j){t,x) has a global maximum at (t,x,p), i.e., {t,x,p) — > 
w{t,x,p,q) + (j){t,x) — {p,p) has a global minimum at {i,x,p). From ()5.28p we get 

-(Pt + H{i, X, -Del), -D^cj), p,q)<0, 

the desired inequality. □ 

Proof of Theorem 15.21 : Following standard arguments, one first checks that 
and V~ are globally Holder continuous, and uniformly Lipschitz continuous with respect 
to p and q. In order to prove other properties of the value functions, let us introduce an 
extended differential game in game with asymmetric information and terminal 

payoff is defined by the dynamics 

dXs = b{s,Xs,Us,Vs)ds + a{s,Xs,Us,Vs)dBs, sG [t,T], 

dZij^s = iij{s,Xs,Us,Vs)ds, (5.30) 

Xt — X, Zij^i — Zjj, 

where (t, x, z) G [0, T] x IR" x IR^*^, with z = (zij), and the terminal gij{x, z) = z^^ + gij{x). 
We denote by V'^ and V~ the upper and lower value of this game. We note that 

V^{t,x,z,p,q) = V^{t,x,p,q) +^ZijPiqj . (5.31) 
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Following the proofs of Proposition 12.1^ one can check that V'^ and V~ are convex in p 
and concave in q. Hence so are and V^. As in Corollarv 13.11 and Proposition \'6.2\ one 
can also show that V~ is a dual supersolution of the HJ equation 

wt + H~ {t, X, z, Dx^zW, D'^w) = 

where, for {t, x, z) € ]R"+", G IR", Cz G IR^"^ and A e 

H~{t,x,z,(,x,(.z,A) = sup^gy infugt/{< b{t,x,u,v),^x > 

+^Tr{Aa{t, x, u, v)a* (t, x, u, v)) + J^-j £ij{t, x, u, v)^z,ij}, 

while is a dual subsolution of the HJ equation 

Wt + H^{t, X, z, Dx^zW, D^w) = 

where 

H^{t,x,z,^x,^z,A) = inf„g[/sup^gv'{< a;, n, f ), > 

+lTr{Aa{t, x, u, v)a* {t, x, u, v)) + X^jj ^ijit, x, u, v)^z,ij}- 
Note that this is precisely at this point that the players have to use the new definition of 
strategies. Indeed, in order to build their strategies in the sub- and superdynamic program- 
ming, they have to compute the running costs Zij (see the proof of Proposition 13. ip . This 
is possible since, at time s, they know the controls u. and v. and the trajectory X. up to 
time s — 6, and therefore can compute Zij^s = Zij + £ij{T, Xr,Ur,Vr)dT. 

Using Lemma [5.3l one can then show that V~ is a dual supersolution of the HJ equation 

Wt + H~{t, X, Dw, D^w,p, q) = 

where 

H~{t,x,C,A,p,q) = sup^£V'™f«eC/{< b{t^ x,u,v), > 

+^Tr{Aa{t, x, u, v)a*{t, x, u, v)) + Yl^j iij{t, x, u, v)piqj} 

while is a dual subsolution of the HJ equation 

Wt + H~^{t, X, Dw, D'^w,p, q) =0 

where 

H~^{t,x,C,A,p,q) = inf„6[/sup„gy{< b{t, x,u,v), > 

+^Tr{Aa{t, x, u, v)a*{t, x, u, v)) + Yjij ^ijit, x, u, v)piqj} 
Finally combining Isaacs' assumption, which states that H := = H~, the fact that H 
satisfies assumption ()4.16p and the comparison principle shows that = V~ is the unique 
dual solution of ([07|) . □ 
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6 Appendix : A maximum principle 

The following result — used in a crucial way in the proof of the comparison principle — is an 
adaptation to our framework of the maximum principle for semicontinuous functions (see 
Theorem 3.2 of |9]): 

Theorem 6.1 (Maximum principle) For k = 1,2, let Ok be open subsets o/ ]R"'= and 
Wk : Ok'x A(/) X A(J) ^ IR 6e such that 

(i) wi = wi{x,p,q) is upper semicontinuous in all variables, convex with respect to p and 
uniformly Lipschitz continuous with respect to q, 

(a) W2 = W2{y,p,q) is lower semicontinuous in all its variables, concave with respect to q 
and uniformly Lipschitz continuous with respect to p, 

(Hi) there is some map (/> : Oi x O2 — > IR and some point {x,y) £ Oi x O2 such that 
the map 

{x,y) max{wi{x,p,q) - W2{y,p,q) - 4>{x,y)} 
p,q 

has a maximum at {x,y). 

Then, for any e > 0, there are {p,q) G A(/) x A(J), {p,q) € IR^^ x IR"^ and {Xi,X2) G 
<Sni X <Sn2 such that the map 

{x,y,P,q) wi{x,p,q) - W2{y,p,q) - 4'{x,y) 

has a maximum at {x,y,p,q), 

p e dpW2{y,p,q), q ^ d^w{{x,p,q) , (6.32) 

{-D,cl){x,y),X^) eD^w{{x,p,q), {Dy(P{x,y), X2) eD^w*2{y,p,q) (6.33) 

and 

with A = D^(/)(x, y). 

Remark : Compared with the classical maximum principle, the additional difficulty 
here is the fact that we need elements of D'^'~w\ and of D'^'~^W2 while we have only infor- 
mation on the behavior of the difference wi — W2 — 4>- 
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Proof of Theorem 16.11 : We follow closely the proof of Theorem 3.2 of [9j. Let us 

start by some reductions: 

Reductions : As in [9], we can assume without loss of generality that Ok = 1R"'=, 
X = y = and (p{x, y) = A{x^ y)-{x, y) and 

max{wi{x,p,q) - W2{y,p,q) - 4>ix,y)} = . (6.35) 

We can also assume that, for any {p',q') G x A(J), 

if {x, y,p', q') is a maximum point of z^i — ^2 — (/>, 

(6.36) 

then {p',q') belongs to the interior of A(/) x A(J). 

Indeed, let us assume that Theorem 16.11 holds true under this additionnal assumption and 
let us prove that it holds true without. Let 

zix, y,p, q) = wi{x,p, q) - W2iy,p, q) - (l){x, y) . 

Among the (p, q) for which z{x, y,p, q) has a maximum, let us choose {po, %) such that the 
total number of indices i and j for which {po)i = or {qo)j = is maximal. Let us denote 
by /' and J' the set of indices i and j for which {po)i > and {qo)j > 0. We then define 
w[, ^'i P'o aiicl (Jq as the natural restriction of wi, W2, z, p^ and % to A(/') and A(J'). 
We note that {x,y,p'Q,qQ) is a maximum point of z on A(/') x A(J') and that 

assumption (j6.36p holds, since otherwise one would have a contradiction with the particular 
choice of {po, %). 

Using now Theorem 16.11 with assumption (j6.36p . we can build {p',q') € A(/') x A(J'), 
q' G d^wi{x,p',q'), p' G dpW2{y,p',q') and {Xi,X2) G x such that (I632D . (I633D 
and (|6.34|) hold. Then we extend {p',q') to {p,q) G A(/) x A(J) by setting pi = p\ for 
i £ I' and pi = otherwise, and qj = qj for j G J' and qj = otherwise. We also extend 
q' to q £ d^wi{x,p, q) and q' to q £ d^wi{x,p, q) by setting qj = M for j G J\J' and 
Pi = —M for i G /\/', where M is a Lipschitz constant of wi and 1112 with respect to q and 
p respectively. This defines p,q,q,p and {Xi,X2) for which (j6.32p . (j6.33p and (|6.34p hold. 

So it remains to prove that Theorem 16.11 holds true under the additional assumption 

Step 1 : introduction of the inf- and supconvolutions. As in |9], we have 

{wi{x',p,q) - ^\x' - xp) - {w2{y',p,q) - ^\y' - yp) < {{A + €A^){x,y), (x,y)) 
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for any {x,x' ,y,y' ,p,q), where A = ^ + \\A\\. Let us set for A' G (0, A), 

'^l ' |2 '^'i / |2n 



wi{x,p,q)= max {wi{x' ,p,q') - -\x' - x\ -— 

x'GlR"i, g'GA(J) 2 2 



and 



' ' \ , ^\ ' |2 , -^'i / \2\ 



W2{y,P,q)= min ^ (w2{y ,p ,q) + -\y - y\ + —\P - P 
?/'eil"2,p'eA{/) z z 



With these definition we have that wi is semiconvex in ah its variables with a modulus A', 
semiconvex in x with a modulus A and convex in p (because wi is convex in p by assumption). 
In the same way, W2 is semiconcave in all its variables with a modulus A', semiconvex in y 
with a modulus A and concave in q (because 102 is concave in q by assumption). Moreover 

wi{x,p,q) -W2{y,p,q) - {{A + eA^){x,y),{x,y)) <0 \/{x,y,p,q) . (6.37) 

Since wi < wi and 1^2 > W2, there are some {p,q) such that equality holds in (j6.37p at 
{0,0,p,q). Furthermore, if equality holds at (0,0,p, g), then (0,0, p,q) is a maximum point 
in (j6.35p and assumption (j6.36p states that {p,q) belongs to the interior of A(J) x A(J). 

Step 2 : use of Jensen maximum principle. Let us now introduce some small pertu- 
bation of the equation: for a > and ( = {(x, Cy, Cpi Cg) ^ ]R"i+"2+/+J^ gg^^ 

Z(:{x,y,P,q) = wi{x,p,q) - W2{y,p,q) - {A + eA'^){x,y).{x,y) 

-adxp + + [p|2 _ |g[2) _ {C^{x,y,p,q)) . 

Note that, because of the penalisation term a(|2;p + |yp), for any rj > 0, we can choose 
7 small enough such that, for any such that \(\ < 7, any maximum of is of the form 
{x,y,p,q) for some {x,y) G B^j. 

Let 7 as above. Since zq is semiconvex, has a maximum at {0,0,p,q), Jensen maximum 
principle (see Lemma A. 3 of [9] for instance) states that the set 



'' {x,y,p,q) £ X A(/) x A(J) , 3C, |C| < 7 , such that 
(i) has a maximum at {x,y,p,q) and 
(a) wi and 11)2 have a derivative at {x,y,p,q) 



has a positive measure. We note that in the quoted Lemma A. 3, the maximum is required 
to be strict ; this assumption is only used in [9j to localize the maximum points, which is 
not needed here. 
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We also note for later use that, if {x, y,p, q) G E^, there is some C = (Cxj Cyj Cpi Cq) with 
ICI ^ 7 such that has a maximum at {x,y,p,q). In particular, this implies that 

q' wi{x,p,q') - W2{y,p, g') - {Cq,Q') 

has a maximum at q. Since W2 is concave in q, wi coincides with its concave hull with 
respect to q at {x,p,q). Hence, if we set q = ^"'^^^'^'^^ then 

wi{x,p,q) +wl{x,p,q) = q.q and q G dfw{{x,p,q) . (6.38) 
In the same way, if we set p = ^"'^^^'P'g) ^ then we have 

W2{x,p,q) +W2{y,p,q) =p.p and pe dpwl{x,p,q) . (6.39) 

Step 3 : measure estimate of a subset of E-y. Let E'^ be the set of points (z, y, q) € E^ 

dw] 
dq 



such that w\ has a second order Taylor expansion at {x,p, ^^(x,p, q)) and W2 has a second 



order Taylor expansion at (y, ^g^{x,p, q),q). Our aim is to show that E'^ has a full measure 



in E^. 



For this we note that E!y = E}^ n E^ where 



^ f {x,y,p,q) G E^ , w\ has a second order Taylor expansion 
I atix,p,^{x,p,q)) 



El 



and 

{x,y,p,q) E E^ , W2 has a second order Taylor expansion 1 
atiy,^{x,p,q),q) J 

It is therefore enough to show that E}^ and E'^ have a full measure in Ej. We only do the 
proof for E}^, the proof for being symmetric. 
Let us set, for any {x,y,p), 

E'y{x,y,p) = {qe A(J) , {x,y,p,q) G E^} 

and 

E}^{x,y,p) ={qe A(J) , {x,y,p,q) G E}^} 

Since E^/ has a positive measure, from Fubini Theorem we have to show that, for any (x, y, p) 
such that the set E^{x,y,p) has a positive measure, the set E}^{x,y,p) has a full measure 
in E^{x,y,p). 
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For this, let us introduce the map 

q>:q^ d^£l^^ defined on E^i x,y,p). We are going 

to show that 

1 

which will imply that 



yquq2&E^{x,y,p), \qi - q2\ < —Mqi) - HQ2)\ , (6.40) 



C EJx,y,p) measurable, C-^ (E) < --\jC\^iE)) , (6.41) 

(2ay 

where denotes the Lebesgue measure in IR-^. Then we will prove that (|6.41|) implies our 
claim. 



Proof of ^6-4U^ : Let (71,^2 ^ E^{x,y,p). There are Ci and ^2 such that z^^. has a 



maximum at {x,y,p,qk) for k = 1,2. The first order optimality conditions imply that 

dw2{y,p,qk) „ , . f 1, 1 o 

^(^fe) = ^aqk + Ck,q for k = 1,2. 

Using again the optimality of at qi and the fact that q W2{y,p, q) is concave, we have 

wiix,p,q2) < wi{x,p,qi) + (^(^ ^'"^(V'P'ik) _ 2aqi + Ci,q^ , (g2 - gi)) - a|g2 - giP 
< wi{x,p,qi) + ($(gi), (92 - qi)) - a\q2 - ^ip 
Reversing the role of qi and q2 gives 

u'i(x,p,gi) < wi{x,p,q2) + (^>(g2), (gi - 92)) - a\q2 - qif 
Adding the two previous inequalities then leads to 

< ($(92) - ^(gi)).(gi - 92) - Mq2 - giP ■ 

Whence (ICTD . 

Proof of ^6.41\ ) : Let £' be a measurable subset of E^{x,y,p). We note that (j6.40p 
states that $ is a bijection between E and its image, with a Lipschitz continuous inverse. 
Hence 



C\E) = C\^-\^{E))) < ^C\^E)) , 



i.e., (f6^ holds. 



We finally show that E}^{x,y,p) has a full measure in E^{x,y,p) for any {x,y,p) such 
that Ey{x,y,p) has a positive measure. Let F be the set of {x,p,q) such that has 
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a second order Taylor expansion at {x,p,q). Since F has a full measure, for almost all 
{x,p) E IR" X A(/), the set F{x,p) = {q £ IR"^ , {x,p,q) G F} has a full measure in 
IR"^. Let {x,p) be such a pair and such that E^{x,y,p) has a positive measure. Then 
^{E.y{x,y,p)) also has a positive measure from ()6.4ip . Since ^{E^{x,y,p))\F{x,p) has a 
zero measure and since 

<l>-^{<l>{E^{x,y,p))\F{x,p)) = E^{x,y,p)\E]{x,y,p) , 

using again (j6.4ip shows that Ey{x,y,p)\E}^{x,y,p) has a zero measure. This completes 
our claim. 

Step 4 : (further) magic properties of sup-convolution. We now explain that one 
can use second order Taylor expansions of w\ and W2 to get elements of D'^'~'w\, D'^'^wi, 
D^^+w^ and D'^'wi. 

^From our assumption ()6.36p . we know that, for e small enough, if {0,0, p,q) realizes 
the equality in (j6.37p . then (p, q) belongs to the interior of A(/) x A(J). Hence we can find 
a, 7 > so small that, for any (" with \(^\ < 7, if (x, y,p, q) realizes the maximum of Z(^, then 
{p,q) belongs to the interior of A(/) x A(J). 

Let us now fix 7 > small enough and let us compute w\ at {x,p, q) for (x, y,p, q) G E'^ 
and q = ^^^igM. We have 

A A' 

w\{x,p,q) = min {q .q+ -\x' - x\^ + —\q'' - q\^) - wi{x ,p,q'')) (6.42) 

^From (j6.38p . we have that w\(x,p, q)^w\{x,p, q) = q.q and q € d'^w\{x,p, q). In particular, 
q' = q is a minimum point in (j6.42p . Since q belongs to the interior of A(J), the optimality 
conditions imply that, if {x',q, g") is a minimum of (|6.42p . then q = q^^ — -^q- Therefore 

w\{x,p,q) = -2^|gp + min^/_g"(g".g - ti;i(x',p,g") + - xp) 
= "wl^P + min^/('u;}(x',p, g) + - xp) , 

In particular, g" € dfwi{x',p,q), which shows that 

q+^qedfw\ix + ^/X,p,q) (6.43) 

Moreover, x — > w\{x,p,q) is equal, up to a constant, to the inf-convolution of w\ with 
respect to x. Since w\ has a second order Taylor expansion in x at such a point {x,p, q), the 
classical "magic properties" of inf-convolution (see Lemma A. 4 of [9]) state that x' = x+^/\ 
and 

{Dw\{x,p,q),D'^w{{x,p,q)) e D'^-w{{x + i/ \,p,q) . (6.44) 
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where ^ = Dw\{x,p,q). 

Following [T] we also note that for any x' close to x, we have 

wiix', p,q) < q.q - w{{x',p,q) = wi{x,p,q) +wl{x,p,q) - wl{x',p,q) 

because wi{x,p,q) + w\{x,p,q) = q.q. Since w\ has a second order Taylor expansion at x, 
this gives 

— {Dw\{x,p,q), D'^w\{x,p,q)) £ D'^'~^wi{x,p,q) . (6.45) 
In a symmetric way, if (x, y,p, q) G E'^ and p = ^^^fap.g) ^ ^j^gj^ 

pu;^(y,l5,g),Z)2ii*(y,]5,(z)) Gl)2,+^^(y + e/A,p,g) , (6.46) 

where ^ = Dw2{y,p, 



P+^pedTw^iy + ^/X,p,q) (6.47) 



and 

- (L»t^2*(y,]5,g),z)2u;2*(y,p,g)) e d^'-Mv.p.q) ■ (6-48) 

Step 5 : conclusion. From the previous steps, we know that the set E!y defined in step 
3 has a positive measure for any a, 7 > sufficiently small. Hence we can find sequences 
K +00, an, 7n 0+, Cn = (C",Cy,Cp,Cg) ^ 0, {xn, Vn, Pn, q-n.) Converging to some 
{0,0, p,q) such that {xn,yn,Pn,Qn) £ E'^ and such that the map has a maximum at 

(•^rii 2/ni Pn, Qn)- 

Let US set 

. _ dw2{y,Pn,qn) . _ gwi(x,Pn,gn) .„x 

(Cl,X") = (i:)w}(x„,p„,g„),Z)^t()}(x„,p„,gn)) 

and 

(?2>^2) = {Dwl{yn,Pn,qn),D'^wl{yn,Pn,qn)) ■ 

^From (j6.43p and (j6.47p we have 

Pn + jfPn e dpW*2{yn + i2/^^Pn,qn) and qn + G (x„ + ^^/A, p„ , ^n) . (6.50) 

^From (|6.45p and (j6.48p we have 

-(er,^r) G I)^'+u'i(a;„,pn,gn) and - (^2,^?) € D'^^-W2{Xn,Pn,qn) , 

Since furthermore (x, y) — > z^^{x,y,pn,qn) has a maximum at {xn,yn,Pn, qn), the first and 
second order optimality conditions imply that 

(-er,e2) = + eA')(x„,y„) + 2a„(x„,y„) + (C,C) (6-51) 
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and 




) 



<A + eA^ + 2anl 



(6.52) 



The left-hand side inequahty is due to the fact that wi and 'W2 are semiconvex and semi- 
concave w.r. to X and y respectively with a modulus A = + ||^||- Using (j6.44p and (j6.46p 
gives 



G D^'-wl{xn + ^'^/X,Pn,qn) and (^2,^2") ^ D^'+wUvn + / X,Pn, Qn) (6.53) 
We now note that (Xf ), (X^), (pn) and {q n) are bounded. For [Xf), {X2) this is an 



obvious consequence of (j6.52p . For (p„) and (g„) this comes from ()6.49p . from the Lipschitz 
continuity assumption of W2 and wi with respect to p and q respectively and from the 
definition of wi and W2- 

We now let n ^ -|-oo. From (j6.5ip . we have (,1,^2 ~^ 0- We can assume that {pn-,qn) — ^ 



(j5,g), Xf ^ Xi and ^ Xg. Then we have from (l630]) . (1633]) and (|632]) that: 

p € dpW2{0,p,q) and g e 9tw}(0,p,g) , 
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